Techniques for preloading and displaying high quality image data

ABSTRACT

A system may have one or more processors and a memory storing instructions, that when executed by the processors, cause the processors to perform operations that include receiving data associated with a user, and determining that the received data corresponds to a direction of movement by the user through a virtual reality (VR) environment, augmented reality (AR) environment, or mixed reality (MR) environment. The operations may also include transmitting a tile of high quality image data to a display device based on the direction of movement by the user, transmitting a command to the display device to display one or more aspects of a region of the VR environment, the AR environment, or the MR environment based on the tile of high quality image data, and preloading one or more additional tiles of high quality image data into a preloader based on the tile of high quality image data.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application Ser.No. 63/002,056, entitled “TECHNIQUES FOR PRELOADING AND DISPLAYING HIGHQUALITY IMAGE DATA”, filed Mar. 30, 2020, which is hereby incorporatedby reference in its entirety for all purposes.

BACKGROUND

The present disclosure relates generally to displaying image data in avirtual reality environment, an augmented reality environment, or amixed reality environment. More specifically, the present disclosurerelates to an imaging system that may preload image data to facilitatedisplaying certain aspects of one or more regions of the virtual realityenvironment, the augmented reality environment, or the mixed realityenvironment on a display device.

As new imaging techniques are developed, higher quality images or videomay be generated and subsequently displayed to a user within a virtualreality (“VR”) environment, an augmented reality (“AR”) environment, ora mixed reality (“MR”) environment via a display device. However, thesehigher quality images or video may have very large file sizes because ofthe amount of data in each image or video. As such, displaying theVR/AR/MR environments provided by such image files or video filesgenerated from high quality imaging techniques at a large scale mayinvolve large amounts of computing resources (e.g., terabytes or more)that may be cost-prohibitive or resource-prohibitive.

This section is intended to introduce the reader to various aspects ofart that may be related to various aspects of the present techniques,which are described and/or claimed below. This discussion is believed tobe helpful in providing the reader with background information tofacilitate a better understanding of the various aspects of the presentdisclosure. Accordingly, it should be understood that these statementsare to be read in this light, and not as admissions of prior art.

SUMMARY

A summary of certain embodiments disclosed herein is set forth below. Itshould be understood that these aspects are presented merely to providethe reader with a brief summary of these certain embodiments and thatthese aspects are not intended to limit the scope of this disclosure.Indeed, this disclosure may encompass a variety of aspects that may notbe set forth below.

In one embodiment, a system may have one or more processors and a memorystoring instructions, that when executed by the processors, cause theprocessors to perform operations that include receiving data associatedwith a user and determining that the received data corresponds to adirection of movement by the user through a virtual reality (VR)environment, augmented reality (AR) environment, or mixed reality (MR)environment. The operations may also include transmitting a tile of highquality image data to a display device based on the direction ofmovement by the user, transmitting a command to the display device todisplay one or more aspects of a region of the VR environment, the ARenvironment, or the MR environment based on the tile of high qualityimage data, and preloading one or more additional tiles of high qualityimage data into a preloader based on the tile of high quality imagedata.

In another embodiment, a method may include receiving, via one or moreprocessors, data associated with a user from one or more input devices,one or more sensors, or both, and determining, via the processors, thatthe received data corresponds to a direction of movement by the userthrough a virtual reality (VR) environment, augmented reality (AR)environment, or mixed reality (MR) environment. The method may alsoinclude transmitting, via the processors, one or more high quality imagedata files to a display device based on the direction of movement by theuser, transmitting, via the processors, a command to the display deviceto display one or more virtual objects of a region of the VRenvironment, the AR environment, or the MR environment based on the oneor more high quality image data files transmitted to the display device,and preloading, via the processors, one or more additional high qualityimage data files into a preloader based on the direction of movement bythe user.

In another embodiment, a non-transitory, computer-readable medium,contains instructions that when executed by one or more processors,cause the processors to perform operations that includes receiving dataassociated with a user from one or more input devices, one or moresensors, or both, and determining that the received data corresponds toa particular direction in a visual field of the user in a virtualreality (VR) environment, augmented reality (AR) environment, or mixedreality (MR) environment. The operations also include transmitting atile of high quality image data to a display device based on theparticular direction in the visual field of the user, transmitting acommand to the display device to display one or more aspects of a regionof the VR environment, the AR environment, or the MR environment basedon the tile of high quality image data, and preloading one or moreadditional tiles of high quality image data into a preloader based onthe tile of high quality image data transmitted to the display device.

Various refinements of the features noted above may exist in relation tovarious aspects of the present disclosure. Further features may also beincorporated in these various aspects as well. These refinements andadditional features may exist individually or in any combination. Forinstance, various features discussed below in relation to one or more ofthe illustrated embodiments may be incorporated into any of theabove-described aspects of the present disclosure alone or in anycombination. The brief summary presented above is intended only tofamiliarize the reader with certain aspects and contexts of embodimentsof the present disclosure without limitation to the claimed subjectmatter.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood when the following detaileddescription is read with reference to the accompanying drawings in whichlike characters represent like parts throughout the drawings, wherein:

FIG. 1 illustrates a block diagram of an imaging system that preloadshigh quality image data corresponding to one or more regions of avirtual reality (VR) environment, an augmented reality (AR) environment,or a mixed reality (MR) environment, in accordance with embodimentsdescribed herein;

FIG. 2 illustrates a schematic diagram of an exemplary VR/AR/MRenvironment, in accordance with embodiments described herein;

FIG. 3 illustrates a block diagram of a preloader and a display deviceof the imaging system of FIG. 1 as a user navigates a first set ofregions of the exemplary VR/AR/MR environment of FIG. 2, in accordancewith embodiments described herein;

FIG. 4 illustrates a block diagram of the preloader and the displaydevice of the imaging system of FIG. 1 as the user navigates a secondset of regions of the exemplary VR/AR/MR environment of FIG. 2, inaccordance with embodiments described herein; and

FIG. 5 illustrates a flowchart of a method for displaying high qualityimage data corresponding to a particular region of a VR/AR/MRenvironment and preloading high quality image data corresponding to oneor more regions bordering the particular region in the preloader, inaccordance with embodiments described herein.

DETAILED DESCRIPTION

One or more specific embodiments of the present disclosure will bedescribed below. In an effort to provide a concise description of theseembodiments, all features of an actual implementation may not bedescribed in the specification. It should be appreciated that in thedevelopment of any such actual implementation, as in any engineering ordesign project, numerous implementation-specific decisions may be madeto achieve the developers' specific goals, such as compliance withsystem-related and business-related constraints, which may vary from oneimplementation to another. Moreover, it should be appreciated that sucha development effort might be complex and time consuming, but wouldnevertheless be a routine undertaking of design, fabrication, andmanufacture for those of ordinary skill having the benefit of thisdisclosure.

When introducing elements of various embodiments of the presentdisclosure, the articles “a,” “an,” “the,” and “said” are intended tomean that there are one or more of the elements. The terms “comprising,”“including,” and “having” are intended to be inclusive and mean thatthere may be additional elements other than the listed elements. One ormore specific embodiments of the present embodiments described hereinwill be described below. In an effort to provide a concise descriptionof these embodiments, all features of an actual implementation may notbe described in the specification. It should be appreciated that in thedevelopment of any such actual implementation, as in any engineering ordesign project, numerous implementation-specific decisions must be madeto achieve the developers' specific goals, such as compliance withsystem-related and business-related constraints, which may vary from oneimplementation to another. Moreover, it should be appreciated that sucha development effort might be complex and time consuming, but wouldnevertheless be a routine undertaking of design, fabrication, andmanufacture for those of ordinary skill having the benefit of thisdisclosure.

As described above, high quality images or video may be generated fromvarious imaging techniques and subsequently displayed to a user within avirtual reality (“VR”) environment, an augmented reality (“AR”)environment, or a mixed reality (“MR”) environment via a display device.Such high quality imaging techniques may include techniques in lightfield technology, point cloud modeling, voxelization, and so forth.Through the use of image files or video files generated from thesetechniques and subsequently displayed to the user, the user may be ableto perceive a level of detail in digitally presented objects in VR/AR/MRenvironments similar to the level of detail typically perceived by theuser in the real world. For example, a light field volume may begenerated by capturing one or more images of a real-world object thatinclude brightness values and color values of each light ray received byan image sensor as well as the direction and/or the angle of each lightray received by the image sensor. In this way, the light field volumemay contain sub-images that slightly differ from each other based on thedirection and/or the angle of each light ray associated with thesub-image. Based on a desired focus depth or a desired perspective ofthe object, the light field volume may be processed to produce atwo-dimensional (“2D”) image or a three-dimensional (“3D”) image of theobject that corresponds to the desired focus depth or the desiredperspective. In this way, images or videos generated from light fieldtechnology may be displayed in VR/AR/MR environments to provide the userwith a similar perspective of an object in the VR/AR/MR environments asthe user would have of the object in the real world.

However, due to the amount of image data in each light field volume,image files or video files providing light field volumes are very large(e.g., a terabyte or more). Similarly, sizes of image files and videofiles generated via point cloud modeling, voxelization, and other highquality imaging techniques may also be very large. As such, use of suchfiles generated from high quality imaging techniques to display VR/AR/MRenvironments at a large scale may involve large amounts of computingresources that may be cost-prohibitive or resource-prohibitive. Forexample, in an interactive ride setting in which a user may move aroundwithin a VR/AR/MR environment and the visual field of the user maychange within the VR/AR/MR environment, multiple high quality imagefiles may be loaded to display a scene based on the user's positionwithin the VR/AR/MR environment, the user's visual field within theVR/AR/MR environment, or both. Additionally, the size of the highquality image files and the number of the high quality image files mayexponentially grow with the size of the VR/AR/MR environment in whichthe user is permitted to move around in.

Accordingly, embodiments of the present disclosure are generallydirected to an imaging system that may preload or cache (e.g., intohigh-speed data storage) one or more tiles of high quality image datathat correspond to regions within a VR/AR/MR environment that a user maypotentially navigate through or perceive from a neighboring region.Preloading a tile of high quality image data may enable faster accessand transfer of the tile when compared to storing the tile in otherstorage devices (e.g., non-volatile memory). After preloading the tilesof high quality image data, the imaging system may cause a displaydevice to display a particular tile of the preloaded tiles (e.g., bytransmitting the particular tile to the display device) based on userinput indicative of a desired direction that the user will proceedthrough the VR/AR/MR environment or that the user is facing within theVR/AR/MR environment. The imaging system may then cause the displaydevice to display one or more aspects of the corresponding region of theVR/AR/MR environment based on the received tile as the user proceeds inthe desired direction toward and within the region of the VR/AR/MRenvironment or as the user faces the region of the VR/AR/MR environmentin the desired direction from a neighboring region. For example, theimaging system may cause the display device to display movie-like scenesor imagery to the user that corresponds to aspects of the region of theVR/AR/MR environment.

As referred to herein, “high quality” image or video data refers to 2Dimage data, 2D video data, 3D image data, 3D video data, or the like,generated via light field technology, point cloud modeling,voxelization, or the like. In some embodiments, the resolution of theVR/AR/MR environment with the high quality image data is at least 4Kpixels (e.g., 3840 pixels×2160 pixels or 4096 pixels×2160 pixels),including 8K pixels (e.g., 7680 pixels×4320 pixels). Additionally, asreferred to herein, a “tile” refers to one or more high quality imagedata files that may be used to present aspects of a particular region ofthe VR/AR/MR environment to the user on a display device after theimaging system transmits the tile to the display device. For example,the VR/AR/MR environment may be divided into a predetermined number ofregions that correspond to respective tiles. Each tile may contain highquality image data that a display device may use to display aspects ofthe corresponding region of the VR/AR/MR environment. In particular, atile may be used to display virtual objects of the VR/AR/MR environment,a virtual space above the user, a virtual space below the user, modifiedportions of the user, or the like, using the display device. In someembodiments, a tile may have at least 1 terabyte (TB) of data, at least850 megabytes (MBs) of data, at least 750 MBs of data, at least 600 MBsof data, at least 500 MBs of data, or the like.

In one embodiment, the imaging system may cause the display device todisplay a city-based VR/AR/MR environment, which the user may navigate(e.g., move) through as part of an interactive ride or experience in anamusement park. At the start of the interactive ride, the imaging systemmay render a tile from a database that corresponds to a default orpredetermined region in the city-based VR/AR/MR environment that theuser is positioned in. The imaging system may also preload one or moretiles from the database that correspond to respective regions (e.g.,neighboring or adjacent regions) in the city-based VR/AR/MR environmentthat the user may potentially navigate through or perceive from thedefault region. The regions that correspond to the preloaded tiles mayborder the default region within the city-based VR/AR/MR environment.The user may be permitted to proceed through such regions after leavingthe bounds of the default region, and/or the user may be permitted toperceive such regions while the user is at or near the bounds of thedefault region. For instance, as the user approaches a virtual ordisplayed street intersection at the bounds of the default region, theimaging system may render the street intersection and a portion of afirst street in a forward direction relative to the user, a portion of asecond street in a leftward direction relative to the user, and/or aportion of the second street in a rightward direction relative to theuser, based on the tile that corresponds to the default region. Thesystem may then preload tiles from the database corresponding torespective regions past the rendered portions of the first street andthe second street.

In response to a user input indicative of a desired direction that theuser will proceed through the intersection, the imaging system may causethe display device to display the preloaded tile that corresponds to theregion (i.e., selected region) in the desired direction through theintersection (e.g., by transmitting the preloaded tile to the displaydevice). As the user proceeds toward the selected region and navigateswithin the selected region, the imaging system may cause the displaydevice to present various aspects of the selected region based on thereceived preloaded tile of high quality image data. Additionally, theimaging system may preload one or more additional tiles from thedatabase that correspond to respective regions in the city-basedVR/AR/MR environment that the user may potentially navigate through(e.g., regions that the user may move to next) or perceive from theselected region. The regions that correspond to the preloaded tiles mayborder the selected region within the city-based VR/AR/MR environment.The user may be permitted to proceed through such additional regionsafter leaving the bounds of the selected region, and/or the user may bepermitted to perceive such additional regions while the user is at ornear the bounds of the selected region. In some embodiments, the imagingsystem may also discard the previously preloaded tiles that correspondto regions in respective directions that the user did not decide totake.

The imaging system may then repeat this process until the user hascompleted the interactive ride. That is, the imaging system may repeat aprocess of preloading one or more tiles of high quality image data froma database that correspond to respective regions in a VR/AR/MRenvironment that the user may potentially navigate through and/orperceive from a neighboring region in the VR/AR/MR environment. Afterpreloading the tiles of high quality image data, the imaging system maycause a display device to display a particular tile of the preloadedtiles based on user input indicative of a desired direction that theuser will proceed through the VR/AR/MR environment or that the user isfacing within the VR/AR/MR environment (e.g., by transmitting theparticular tile to the display device). The imaging system may cause thedisplay device to display corresponding aspects of the region of theVR/AR/MR environment based on the received tile as the user proceeds inthe desired direction toward and within the region of the VR/AR/MRenvironment and/or as the user faces the region of the VR/AR/MRenvironment in the desired direction from a neighboring region. In thisway, the imaging system may process and transmit a smaller amount ofhigh quality image data at multiple times during the interactive ride,rather than processing and loading an entire high quality image dataset(e.g., corresponding to a large number of tiles of high quality imagedata) at one time. As such, the techniques described herein continuallyreduce and/or optimize the amount of computing resources that theimaging system uses to provide the user with a seamless experience ofthe VR/AR/MR environment based on user input indicative of a desireddirection of user navigation through the VR/AR/MR environment and/or adesired direction of a user's visual field in the VR/AR/MR environment.

By way of introduction, FIG. 1 is a block diagram of an imaging system100 that may preload one or more tiles of high quality image data thatmay be used to display particular aspects of one or more respectiveregions of a VR/AR/MR environment on a display device 108 accessible bya user, in accordance with embodiments described herein. For example, acontrol system 102 of the imaging system 100 may retrieve one or moretiles of high quality image data from a database 104 communicativelycoupled to the control system 102 via a network 106 and store the tilesin a preloader 103 of the control system 102. The preloader 103 may beany suitable high speed data storage that may preload or cache the tilesof high quality image data from the database 104. For example, thepreloader 103 may include a caching device, such as a computerprocessing unit (CPU) cache device. In some embodiments, the CPU cachedevice may include L1, L2, or L3 caches. Preloading a tile of highquality image data into the preloader 103 may enable faster access andtransfer of the tile when compared to storing the tile in other storagedevices (e.g., non-volatile memory). For example, the preloader 103 mayhave a data transfer speed of at least 10 gigabytes per second (GB/s).In some embodiments, the preloader 103 may have a data transfer speed ofat least 25 GB/s, at least 250 GB/s, or at least 1 terabyte per second(TB/s), or the like. Additionally, the preloader 103 may have anysuitable size to store an appropriate number of preloaded tiles. In someembodiments, the preloader 103 may have a data storage size of at least5 TBs, at least 2 TBs, or at least 1 TB, or the like.

The control system 102 may also include a memory 105 and a processor107. The processor 107 of the control system 102 may include one or moreof any suitable type of computer processor or microprocessor capable ofexecuting computer-executable code, including but not limited to one ormore field programmable gate arrays (FPGAs), application-specificintegrated circuits (ASICs), programmable logic devices (PLDs),programmable logic arrays (PLAs), and the like. The processor 105 may,in some embodiments, include multiple processors. The memory 105 mayinclude any suitable articles of manufacture that serve as media tostore processor-executable code, data, and the like. The memory 105 maystore non-transitory processor-executable code used by the processor 107to perform the presently disclosed techniques.

As described above, each tile of high quality image data may begenerated via any suitable high quality imaging technique, such as lightfield technology, point cloud modeling, voxelization, or the like. Aftera tile of high quality image data has been generated, the tile may bestored in the database 104. In some embodiments, the tile may be indexedin the database 104 with a tile identifier, a region identifierassociated with the corresponding region of the VR/AR/MR environment,one or more bordering region identifiers associated with correspondingbordering regions of the VR/AR/MR environment, or the like. The controlsystem 102 may transmit a request to the database 104 with indicationsof (e.g., pointers to) the tile identifier, the region identifier,and/or the bordering region identifiers, and the database 104 maytransmit a response with the corresponding tiles to the control system102.

The display device 108 may be any suitable device for displayingVR/AR/MR content to the user, such as smart glasses, a virtual retinaldisplay, one or more contact lenses, a computer, a mobile device, a headmounted device, or the like. The display device 108 may optionally haveone or more sensors 110 that may acquire data associated with the userand transmit the data, via the network 106, to the control system 102for analysis. The control system 102 may use the received dataassociated with the user to determine a direction in which the user islooking (e.g., a particular direction in the user's visual field) or adirection that the user is intending to move (e.g., a desireddirectional movement of the user). For example, the sensors 110 mayinclude one or more image sensors that may acquire image data or videodata associated with the user's eyes, the user's head, the user's limbs,or the like, one or more microphones that may acquire sound dataassociated with the user, one or more motion sensors (e.g., a velocitysensor, a position sensor, or an accelerometer) that may acquire motiondata associated with the user, or the like. In some embodiments, thesensors 110 may be alternatively or additionally attached to the user'sbody. For example, one or more motion sensors may be disposed on theuser's hands, wrists, arms, fingers, legs, feet, torso, or any othersuitable body part of the user to acquire motion data of the user. Thesensors 110 may alternatively or additionally be disposed in thephysical environment of the user. For example, in an interactive ridesetting, the sensors 110 may be disposed along a predetermined path theuser may physically walk along or within a ride vehicle associated withthe user.

After acquiring data associated with the user, the sensors 110 maytransmit the data associated with the user to the control system 102 viathe network 106 for analysis to determine different types of user inputthat may be provided by the user to modify the VR/AR/MR environment orotherwise control the user's experience of the VR/AR/MR environment. Insome embodiments, the control system 102 may determine a desireddirectional movement of the user through the VR/AR/MR environment basedon an analysis of the data associated with the user, or a desireddirection in the visual field of the user in the VR/AR/MR environmentbased on an analysis of data received from the display device 108. Forexample, the control system 102 may determine one or more usercharacteristics, such as a user position (e.g., of the user's eyes,arms, legs, head, or body), a user movement (e.g., of the user's eyes,arms, legs, head, or body), or a user orientation (e.g., of the user'seyes, arms, legs, head, or body (such as a directional tilt, a pitch, ayaw, or a roll)) based on the analysis of the data received from thedisplay device 108. The control system 102 may then determine thedesired directional movement of the user or the desired direction in thevisual field of the user based on the determined user characteristics.In some embodiments, the control system 102 may compare the determineduser characteristics to one or more stored, learned, or otherwiseinterpretable directional movements of the user or desired directions inthe visual fields of the user in a memory accessible by the controlsystem 102. The control system 102 may also use image and/or patternrecognition techniques to determine the user position, the usermovement, or the user orientation based on the analysis of the datareceived from the display device 108. For example, the image and/orpattern recognition techniques or algorithms may include machinelearning, artificial intelligence, deep learning, convolutional neuralnetworks, or the like. A memory accessible by the control system 102 maystore an image recognition model, a speech recognition model, or thelike. Such models may be trained by inputting sample data (e.g., images)of people, such as the user, and indications of user position, usermovement, or user orientation in the data. After such models aretrained, the control system 102 may then use one or more of the modelsto determine a particular user position, a particular user movement, ora particular user orientation in data received from the display device108. The control system 102 may then determine that such determined usercharacteristics are associated with a desired directional movement ofthe user or a desired direction in the visual field of the user.

Additionally or alternatively, the control system 102 may receive userinput or commands, such as a gesture command from the user, a voicecommand from the user, or the like, from one or more user input devices112. For example, the control system 102 may analyze the user input orcommands and determine a gesture command or a voice command via imageand/or pattern recognition techniques or algorithms, including machinelearning, artificial intelligence, deep learning, convolutional neuralnetworks, or the like. For example, a memory accessible by the controlsystem 102 may store an image recognition model, a speech recognitionmodel, or the like. Such models may be trained by inputting sample data(e.g., images or speech) of people, such as the user, and indications ofvarious gesture commands or voice commands in the data. After suchmodels are trained, the control system 102 may then use one or more ofthe models to determine a particular gesture command or a particularvoice command in user input received from the user input devices 112.

Based on the determined user characteristics or user commands, thecontrol system 102 may modify the VR/AR/MR environment perceived by theuser accordingly. For example, the control system 102 may cause thedisplay device 108 to display a tile of high quality image data from thepreloader 103 based on movement of the user in a physical environmentafter receiving the tile from the control system 102. The tile of highquality image data may correspond to a region of the VR/AR/MRenvironment that borders the current region of the VR/AR/MR environmentin which the user is virtually located. As the user approaches thebounds of the current region in the VR/AR/MR environment, the controlsystem 102 may cause the display device 108 to present aspects of theregion bordering the current region based on the received tile of highquality of image data to provide the user with a continuous experienceof the VR/AR/MR environment as the user would experience in the realworld. That is, the control system 102 may cause the display device 108to gradually display certain aspects of the bordering region of theVR/AR/MR environment to the user as the user proceeds toward thebordering region of the VR/AR/MR environment.

Based on the determined user characteristics or user commands, thecontrol system 102 may also preload one or more tiles from the database104 that the user may potentially navigate through (e.g., regions thatthe user may move to next) or perceive from a region that the user hasvirtually moved into. For example, the control system 102 may determinethat the user is intending to virtually move in a desired directiontoward and/or into a particular region in the VR/AR/MR environment basedon the determined user characteristics or commands. Based on the desireddirection of the user, the control system 102 may preload one or moretiles that correspond to regions that respectively border the particularregion in the VR/AR/MR environment that the user is intending to movetoward and/or into.

Additionally, the control system 102 may determine to modify the user'svisual field (e.g., viewing angle) of the VR/AR/MR environment displayedvia the display device 108 based on the determined user characteristicsor user commands. For instance, based on the position of the user's eyesor the movement of the user's eyes, the control system 102 may modifythe appearance of the high quality image data displayed via the displaydevice 108 to the user such that the user may perceive the VR/AR/MRenvironment with a similar visual field as the user would have in thereal world. That is, in response to the user changing the user's viewingangle of an object within the VR/AR/MR environment, the control system102 may modify the appearance of the object in the VR/AR/MR environmentsimilar to how the object would appear in the real world to the userwith a similar change in viewing angle, and cause the display device 108to display the modified appearance.

As shown in the illustrated embodiment, the control system 102 may becommunicatively coupled to one or more user input devices 112 associatedwith the user. For example, the user input devices 112 may include ajoystick, a steering wheel, a touchscreen display; one or more inputdevices of a mobile phone, or any other suitable device for providinguser input. In some embodiments, the user input devices 112 may becommunicatively coupled to the display device 108. In any case, thecontrol system 102 may receive one or more user commands from the userinput devices 112. After receiving the user commands from the user inputdevices 112, the control system may modify the VR/AR/MR environment,adjust the user's field of vision of the VR/AR/MR environment, orotherwise control the user's experience of the VR/AR/MR environmentbased on the received user command. For example, the control system 102may compare the received user command to one or more stored, learned, orotherwise interpretable user commands in a memory accessible by thecontrol system 102, and modify the VR/AR/MR environment based on thecomparison. In some embodiments, the memory may include the memory 105,a read-only memory (ROM) of the control system 102, or the database 104.

It should be noted that any suitable network 106 may be employed in theembodiments described herein. For instance, the network 106 may includeany wired or wireless communication network implemented, such as a localarea network (LAN), a wide area network (WAN), and the like. The network106 may enable wired or wireless communication via any suitablecommunication protocol, such as Wi-Fi, mobile telecommunicationsprotocols (e.g., 2G, 3G, 4G, 5G, Long-Term Evolution (LTE), New Radio(NR)), Bluetooth®, near-field communications protocols, and the like.

With the foregoing in mind, FIG. 2 illustrates a schematic diagram 200of an exemplary VR/AR/MR environment that a user may experience in aninteractive ride setting, in accordance with embodiments describedherein. In the illustrated embodiment, the schematic diagram 200 of theVR/AR/MR environment may be designed as a city in which the user maynavigate between multiple regions of the VR/AR/MR environment (i.e.,202, 204, 206, 208, 210, 212, 214, 216, 218, 220, 222, 224, 226, 228) atdesignated intersections (i.e., A, B, C, D, E, F, G, H, I, J, K) betweenthe neighboring regions. At the beginning of the interactive ride, theuser may begin at a default position 201 in a default region 202 withinthe VR/AR/MR environment. Although the VR/AR/MR environment is describedas a city-based VR/AR/MR environment, it should be understood that suchan embodiment is intended to be exemplary and non-limiting. In otherembodiments, the VR/AR/MR environment may have any other suitabledesign, such as a maze, a castle, a forest, outer space, or the like,that may be divided into multiple regions that correspond to tiles ofhigh quality image data. In any case, an intersection in the VR/AR/MRenvironment may prompt the user to make a decision as to the next regionthe user will proceed to after passing through the intersection. Forexample, each intersection may be presented to the user in the VR/AR/MRenvironment as the user approaches the bounds of the region the user iscurrently present within. As the user approaches the intersection, theuser may provide one or more types of user input to indicate a desireddirection that the user will take through the intersection. As describedabove, the user input may include a physical movement of the user in thedesired direction, a gesture command, a voice command, or the like.

During a suitable period of time before the start of the interactiveride or after the start of the interactive ride, the control system 102of the imaging system 100 may cause the display device 108 to display atile of high quality image data from the preloader 103 that correspondsto the default region 202 in the city-based VR/AR/MR environment thatthe user is positioned in (e.g., by transmitting the tile to the displaydevice 108). For instance, after the display device 108 receives thetile of high quality image data, the control system 102 may cause thedisplay device 108 to display various aspects of the default region 202of the city-based VR/AR/MR environment to the user based on the tile ofhigh quality image data. Such aspects of the default region 202 mayinclude buildings, vehicles, streets, sidewalks, storefronts, the sky,or the like. Also during this period of time, the control system 102 maypreload one or more additional tiles of high quality image data from thedatabase 104 into the preloader 103 that correspond to respectiveregions 204, 206, 208 in the city-based VR/AR/MR environment that theuser may potentially proceed into or perceive from the default region202. That is, the control system 102 may preload regions 204, 206, 208that border, neighbor, and/or are adjacent to the default region 202within the city-based VR/AR/MR environment.

In the illustrated embodiment, as the user approaches intersection A atthe bounds of the default region 202, the control system 102 may causethe display device 108 to display the intersection A and a portion of afirst street in the forward direction relative to the user (i.e.,leading to region 206), a portion of a second street in a leftwarddirection relative to the user (i.e., leading to region 204), and aportion of the second street in a rightward direction relative to theuser (i.e., leading to region 208) based on the tile of high qualityimage data that corresponds to the default region 202. In someembodiments, as the user approaches the bounds of the default region 202or proceeds into the intersection A, the control system 102 may alsocause the display device 108 to gradually display portions of theregions 204, 206, 208 to the user based on corresponding tiles of highquality image data received from the control system 102 to provide theuser with a continuous experience of the VR/AR/MR environment. Forinstance, as the user proceeds towards the center of the intersection A,the control system 102 may cause the display device 108 to displayincreasing portions of the regions 204, 206, 208 along the first streetand the second street based on the distance the user is from the centerof the intersection, the visual field of the user (e.g., the directionthe user is facing), or the like.

Additionally, the user may be permitted to proceed forward within eachregion (e.g., as indicated by a single sided arrow), proceed backwardwithin each region (e.g., as indicated by a single sided arrow), or both(e.g., as indicated by a double sided arrow), with respect to thedefault position 201 of the user in the VR/AR/MR environment. In theillustrated embodiment, for example, the user may be permitted toproceed forward through the region 202 from the default position 201 tothe intersection A while the user may be permitted to proceed forwardthrough the region 206 from the intersection A to intersection C and/orbackward through the region 206 from intersection C to intersection Insome embodiments, the user may be limited to one or more directions ofmovement based on a storyline of the VR/AR/MR environment. Further,although the schematic diagram 200 of the VR/AR/MR environment isillustrated as a grid, it should be understood that such an embodimentis intended to be exemplary and non-limiting. For example, at a decisionpoint (e.g., an intersection), the user may have more than three or lessthan three options regarding the user's direction of movement throughthe decision point. In some embodiments, the user may be permitted tomove upward or downward (e.g., virtual stairs or a virtual ramp) inaddition to moving leftward, rightward, forward, or backward.

With the foregoing in mind, FIG. 3 is a block diagram 300 of thepreloader 103 of the control system 102 and the display device 108, inaccordance with embodiments described herein. Referring to the exampledescribed in FIG. 2 above, as the user 304 approaches intersection Afrom the default region 202 of the VR/AR/MR environment 302, the controlsystem 102 may cause the display device 108 to display aspects of thedefault region 202 of the VR/AR/MR environment 302 based on the tile 308of high quality image data received from the preloader 103 via thecontrol system 102. Additionally, the control system 102 may store tiles(e.g., 310, 312, 314) of high quality image data in the preloader 103from the database 104 that respectively correspond to regions 204, 206,208 of the VR/AR/MR environment 302 that the user may potentiallynavigate through (e.g., regions that the user may move to next) orperceive from the default region 202.

In response to a user input indicative of a forward direction 306through the intersection A, the control system 102 may transmit the tile308 of high quality image data from the preloader 103 to the displaydevice 108. The tile 308 of high quality image data corresponds toregion 206 of the VR/AR/MR environment that the user will proceed towardin the forward direction from the intersection A. Additionally, thecontrol system 102 may preload one or more additional tiles (e.g., 306,308, 310) of high quality image data that respectively correspond toregions 214, 216, 218 (e.g., adjacent regions) of the VR/AR/MRenvironment that the user 404 may potentially navigate through orperceive from the region 206. For example, the control system 102 mayreceive the additional tiles 306, 308, 310 from the database 104 andstore the additional tiles 306, 308, 310 in the preloader 103. In someembodiments, the preloader 103 may discard the unused tiles 310, 314that correspond to the regions 204, 208 of high quality image dataremaining in the preloader 103 before, while, or after receiving theadditional tiles 306, 308, 310. That is, the preloader 103 may discardthe tiles 310, 314 of high quality image data that were not transmittedto the display device 108 before, while, or after receiving theadditional tiles 306, 308, 310. In other embodiments, the preloader 103may retain certain tiles of high quality image data that correspond toregions 204, 208 that neighbor the region 206 that the user chose toproceed toward and/or based on whether the user is permitted to proceedbackward from region 206 to those regions 204, 208. In some embodiments,the preloader 103 also stores tiles corresponding to the regions thatthe user frequently visits. For example, tiles corresponding to avirtual shop, save point, starting point, and so on, may be stored inthe preloader 103, additionally or alternatively to the tilescorresponding to the adjacent regions. In this way, the control system102 of the imaging system 100 may minimize the retrieval of repetitiousdata from the database 104.

As the user proceeds toward the bounds of region 202, the control system102 may cause the display device 108 to display one or more aspects ofthe region 206 based on the tile 312 of high quality image data receivedfrom the preloader 103. In this way, the control system 102 may causethe display device 108 to present certain aspects of the region 206bordering the region 202 to provide the user with a continuousexperience of the VR/AR/MR environment as the user would experience inthe real world. That is, the control system 102 may cause the displaydevice 108 to gradually display aspects of the region 206 to the user asthe user proceeds toward the region 206 of the VR/AR/MR environment. Insome embodiments, the control system 102 may cause the display device108 to display certain aspects of the region 202 based on the tile 308of high quality image data until the user has proceeded a thresholddistance into the region 206. For instance, the threshold distance maycorrespond to a real world distance or a theoretical distance that theuser may travel before such aspects would typically disappear from sightin the real world. In this way, the control system 108 may simulate thesensory experience that the user would typically receive in the realworld. Additionally, in some embodiments, the display device 108 mayretain the tile 308 of high quality image data corresponding to theregion 202 until the user has proceeded into a region (e.g., 218, 220,222, 224, 226, or 228) that is not neighboring the region 202 (e.g., forfaster and more efficient rendering in case the user returns to theregion 202).

Similarly, FIG. 4 is a block diagram 400 of the preloader 103 of thecontrol system 102 and the display device 108 as the user 404 approachesintersection F from the region 218 of the VR/AR/MR environment 402, inaccordance with embodiments described herein. As the user 404 approachesintersection F, the control system 102 may cause the display device 108to display aspects of the region 218 of the VR/AR/MR environment 402based on the tile 408 of high quality image data received from thepreloader 103 via the control system 102. Additionally, the preloader103 of the control system 102 may store a tile 410 of high quality imagedata that corresponds to region 222 of the VR/AR/MR environment that theuser may potentially navigate through (e.g., a region that the user maymove to next) or perceive from the region 218.

In response to a user input indicative of a forward direction 406 thatthe user will take through the intersection F, the control system 102may transmit the tile 410 of high quality image data from the preloader103 to the display device 108. The tile 410 of high quality image datacorresponds to region 222 of the VR/AR/MR environment that the user willproceed toward in the forward direction from the intersection F.Additionally, the control system 102 may preload tile 412 of highquality image data that corresponds to region 228 of the VR/AR/MRenvironment that the user 404 may potentially navigate through orperceive from the region 222. As the user proceeds toward the bounds ofthe region 218, the control system 102 may cause the display device 108to begin displaying one or more aspects of the region 222 based on thetile 410 of high quality image data received from the preloader 103.Because the user is not permitted to proceed backward in region 222toward the intersection F, the control system 102 may discard any copiesof tile 408 in the preloader 103 that corresponds to the region 218.

With the foregoing in mind, FIG. 5 illustrates a flow chart of a method500 for preloading one or more tiles of high quality image data thatrespectively correspond to regions within a VR/AR/MR environment that auser may potentially navigate through or perceive from a neighboringregion in the VR/AR/MR environment, and instructing a display device todisplay a preloaded tile of high quality image data for displayingaspects of a corresponding region of the VR/AR/MR environment based onuser input indicative of a desired direction that the user will proceedthrough the VR/AR/MR environment or that the user is facing within theVR/AR/MR environment, in accordance with embodiments described herein.Although the following description of the method 500 is described in aparticular order, it should be noted that the method 500 is not limitedto the depicted order, and instead, the method 500 may be performed inany suitable order. Indeed, at least some steps of the method 500 may beskipped altogether. Moreover, although the method 500 is described asbeing performed by the control system 102 of the imaging system 100, itshould be noted that it may be performed by any suitable computingdevice.

As described above, a user may be navigating through a region of aVR/AR/MR environment during an interactive ride in an amusement park.Referring now to FIG. 5, at block 502, the control system 102 of theimaging system 100 may receive data associated with the user from one ormore input devices 112, one or more sensors 110, or both, as the userapproaches an intersection at or near the bounds of the region the useris navigating through. For example, the user input devices 112, thesensors 110, or both, may continuously acquire the data associated withthe user and transmit the data to the control system 102. After thecontrol system 102 receives the data associated with the user from theinput devices 112, the sensors 110, or both, at block 504, the controlsystem 102 may determine that the received data corresponds to a userinput indicative of a desired direction that the user is intending toproceed through the intersection. For example, the data associated withthe user may include image data or video data associated with the user'seyes, the user's head, the user's limbs, or the like, sound dataassociated with the user, motion data associated with the user, or thelike. The control system 102 may determine one or more usercharacteristics, such as a position of the user's eyes, arms, legs,head, or body, a movement of the user's eyes, arms, legs, head, or body,or an orientation of the user's eyes, arms, legs, head, or body (e.g., adirectional tilt, a pitch, a yaw, or a roll), or user commands, such asa gesture command from the user, a voice command from the user, or thelike, based on an analysis of the received data associated with theuser. The control system 102 may then determine the desired directionthat the user intends to proceed through the intersection based on oneor more user characteristics, and/or one or more user commands. Forexample, the desired direction may be a leftward direction, a rightwarddirection, a forward direction, an upward direction, a downwarddirection, or the like. In some embodiments, the control system 102 maydetermine the desired direction via any suitable image recognitiontechniques or algorithms, including machine learning, artificialintelligence, deep learning, convolutional neural networks, or the like.

After determining that the received data corresponds to a user inputindicative of the desired direction that the user is intending toproceed through the intersection, at block 506, the control system 102may transmit a tile of high quality image data from the preloader 103 tothe display device 108 based on the user input indicative of the desireddirection. For example, the tile of high quality image data maycorrespond to the region in the desired direction through theintersection. As the user proceeds toward the region in the desireddirection and navigates within the region, the control system 102 maysend a command to cause the display device 108 to present variousaspects of the region based on the received tile of high quality imagedata. For example, the control system 102 may cause the display device108 to display buildings, vehicles, streets, sidewalks, storefronts, thesky, or the like, using the tile of high quality image data. In someembodiments, the control system 102 may cause the display device 108 todisplay one or more videos or animations that correspond to a storylineassociated with the interactive ride.

At block 508, the control system 102 may preload one or more additionaltiles of high quality image data into the preloader 103 based on thetile of high quality image data transmitted to the display device 108 atblock 506. For example, the additional tiles of high quality image datamay respectively correspond to regions of the VR/AR/MR environment thatthe user may potentially proceed toward or navigate through from theregion in the desired direction through the intersection. In someembodiments, the control system 102 may transmit a request for theadditional tiles to the database 104. The request may include one ormore identifiers associated with the tile transmitted to the displaydevice 108 at block 506, the region corresponding to the tiletransmitted to the display device 108, and/or one or more regions thatborder the region corresponding to the tile transmitted to the displaydevice 108. In response to receiving the request for the additionaltiles, the database 104 may transmit a response with the additionaltiles to the control system 102. The control system 102 may then storethe additional tiles received from the database 104 in the preloader103. In some embodiments, the control system 102 may discard unusedtiles of high quality image data present in the preloader 103 beforestoring the additional tiles in the preloader 103.

After preloading the additional tiles into the preloader 103, thecontrol system 102 may repeat the method described above with regard toblocks 502 to 508. That is, the control system may repeat blocks 502 to508 until the user has completed the interactive ride. In this way, thecontrol system 102 of the imaging system 100 may process and transmit asmaller amount of high quality image data at multiple times during theinteractive ride, rather than processing and loading an entire highquality image dataset at one time. As such, the techniques describedherein continually reduce and optimize the amount of computing resourcesthat the imaging system 100 uses to provide the user with a seamlessexperience of the VR/AR/MR environment based on user input indicative ofa desired direction of user navigation through the VR/AR/MR environment.

While only certain features of the disclosure have been illustrated an ddescribed herein, many modifications and changes will occur to thoseskilled in the art. It is, therefore, to be understood that the appendedclaims are intended to cover all such modifications and changes as fallwithin the true spirit of the disclosure.

The techniques presented and claimed herein are referenced and appliedto material objects and concrete examples of a practical nature thatdemonstrably improve the present technical field and, as such, are notabstract, intangible, or purely theoretical. Further, if any claimsappended to the end of this specification contain one or more elementsdesignated as “means for [perform]ing [a function] . . . ” or “step for[perform]ing [a function] . . . ”, it is intended that such elements areto be interpreted under 35 U.S.C. 112(f). However, for any claimscontaining elements designated in any other manner, it is intended thatsuch elements are not to be interpreted under 35 U.S.C. 112(f).

1. A system, comprising: one or more processors; and a memory,accessible by the one or more processors, storing instructions that,when executed by the one or more processors, cause the one or moreprocessors to perform operations comprising: receiving data associatedwith a user from at least one or more input devices or one or moresensors; determining that the received data corresponds to a directionof movement by the user through a virtual reality (VR) environment,augmented reality (AR) environment, or mixed reality (MR) environment;transmitting a tile of high quality image data to a display device basedon the direction of movement by the user; transmitting a command to thedisplay device to display one or more aspects of a region of the VRenvironment, the AR environment, or the MR environment based on the tileof high quality image data; and preloading one or more additional tilesof high quality image data into a preloader based on the tile of highquality image data transmitted to the display device.
 2. The system ofclaim 1, wherein the one or more additional tiles of high quality imagedata are associated with one or more neighboring regions of the VRenvironment, the AR environment, or the MR environment that border theregion of the VR environment, the AR environment, or the MR environment.3. The system of claim 1, wherein the tile of high quality image dataand the one or more additional tiles of high quality image data aregenerated via light field technology, point cloud modeling,voxelization, or any combination thereof.
 4. The system of claim 1,wherein the data associated with the user is received before the userapproaches, virtually, a boundary of the region of the VR environment,the AR environment, or the MR environment.
 5. The system of claim 1,wherein the operations comprise: receiving additional data associatedwith the user from the one or more input devices, the one or moresensors, or both; and determining that the received additional datacorresponds to a second direction of movement by the user through the VRenvironment, the AR environment, or the MR environment.
 6. The system ofclaim 5, wherein the operations comprise transmitting a particular tileof the one or more additional tiles of high quality image data from thepreloader to the display device based on the second direction ofmovement by the user, and transmitting a second command to the displaydevice to display one or more additional aspects of a second region ofthe VR environment, the AR environment, or the MR environment based onthe particular tile of the one or more additional tiles.
 7. The systemof claim 1, wherein the operations comprise discarding one or more tilesof high quality image data not transmitted to the display device beforepreloading the one or more additional tiles of high quality image data.8. The system of claim 1, wherein the operations comprise: determiningthat the received data corresponds to a particular direction in a visualfield of the user in the region of the VR environment, the ARenvironment, or the MR environment; and transmitting a particular tileof the one or more additional tiles of high quality image data from thepreloader to the display device based on the particular direction in thevisual field of the user, and transmitting a second command to thedisplay device to display one or more additional aspects of a secondregion that borders the region in the direction of the visual field ofthe VR environment, the AR environment, or the MR environment based onthe particular tile of the one or more additional tiles of high qualityimage data.
 9. A method, comprising: receiving, via one or moreprocessors, data associated with a user from at least one or more inputdevices or one or more sensors; determining, via the one or moreprocessors, that the received data corresponds to a direction ofmovement by the user through a virtual reality (VR) environment,augmented reality (AR) environment, or mixed reality (MR) environment;transmitting, via the one or more processors, one or more high qualityimage data files to a display device based on the direction of movementby the user; transmitting, via the one or more processors, a command tothe display device to display one or more virtual objects of a region ofthe VR environment, the AR environment, or the MR environment based onthe one or more high quality image data files transmitted to the displaydevice; and preloading, via the one or more processors, one or moreadditional high quality image data files into a preloader based on thedirection of movement by the user.
 10. The method of claim 9, whereinthe one or more additional high quality image data files are associatedwith one or more neighboring regions of the VR environment, the ARenvironment, or the MR environment that border the region of the VRenvironment, the AR environment, or the MR environment.
 11. The methodof claim 9, wherein the one or more high quality image data files andthe one or more additional high quality image data files are generatedvia light field technology, point cloud modeling, voxelization, or acombination thereof.
 12. The method of claim 9, wherein the dataassociated with the user is received before the user approaches,virtually, a boundary of the region of the VR environment, the ARenvironment, or the MR environment.
 13. The method of claim 9,comprising: receiving, via the one or more processors, additional dataassociated with the user from the one or more input devices, the one ormore sensors, or both; and determining, via the one or more processors,that the received additional data corresponds to a second direction ofmovement by the user through the VR environment, the AR environment, orthe MR environment.
 14. The method of claim 13, comprising transmittinga particular high quality image data file of the one or more additionalhigh quality image data files from the preloader to the display devicebased on the second direction of movement by the user, and transmittinga second command to the display device to display one or more additionalaspects of a second region of the VR environment, the AR environment, orthe MR environment based on the particular high quality image data fileof the one or more additional high quality image data files.
 15. Anon-transitory, computer-readable medium, comprising instructions that,when executed by one or more processors, cause the one or moreprocessors to perform operations comprising: receiving data associatedwith a user from one or more input devices, one or more sensors, orboth; determining that the received data corresponds to a particulardirection in a visual field of the user in a virtual reality (VR)environment, augmented reality (AR) environment, or mixed reality (MR)environment; transmitting a tile of high quality image data to a displaydevice based on the particular direction in the visual field of theuser; transmitting a command to the display device to display one ormore aspects of a region of the VR environment, the AR environment, orthe MR environment based on the tile of high quality image data; andpreloading one or more additional tiles of high quality image data intoa preloader based on the tile of high quality image data transmitted tothe display device.
 16. The non-transitory, computer-readable medium ofclaim 15, wherein the operations comprise: receiving additional dataassociated with the user from the one or more input devices, the one ormore sensors, or both; and determining that the received additional datacorresponds to a second direction in the visual field of the user in theVR environment, the AR environment, or the MR environment.
 17. Thenon-transitory, computer-readable medium of claim 16, wherein theoperations comprise transmitting a particular tile of the one or moreadditional tiles of high quality image data from the preloader to thedisplay device based on the second direction in the visual field of theuser, and transmitting a second command to the display device to displayone or more additional aspects of a second region of the VR environment,the AR environment, or the MR environment based on the particular tileof the one or more additional tiles.
 18. The non-transitory,computer-readable medium of claim 15, wherein the one or more additionaltiles of high quality image data are associated with one or moreneighboring regions of the VR environment, the AR environment, or the MRenvironment that border the region of the VR environment, the ARenvironment, or the MR environment.
 19. The non-transitory,computer-readable medium of claim 15, wherein the data associated withthe user is received before the user approaches, virtually, a boundaryof the region of the VR environment, the AR environment, or the MRenvironment.
 20. The non-transitory, computer-readable medium of claim15, wherein the tile of high quality image data and the one or moreadditional tiles of high quality image data are generated via lightfield technology, point cloud modeling, voxelization, or a combinationthereof.